Building Apps & Bots for Social Science Research

Chris Bail
Duke University

Part I: APPS

Salganik's list of the weaknesses of digital trace data:

 

Incomplete
Inaccessible
Non-Representative
Drifting
Algorithmic Confounding
Dirty
Sensitive

But what are the alternatives?

 

But what are the alternatives?

 

-Survey response rates continue to drop

But what are the alternatives?

 

-Survey response rates continue to drop

-Many of the most important questions require longitudinal/relational/qualitative data

But what are the alternatives?

 

-Survey response rates continue to drop

-Many of the most important questions require longitudinal/relational/qualitative data

-Digital trace data have a number of major advantages that conventional sources do not (big, always on, non-reactive)

Can we have our cake and eat it too?

Apps can Address Many Limitations

 

Incomplete
Inaccessible
Non-Representative
Drifting
Algorithmic Confounding
Dirty
Sensitive

Example: The Viralgorithm

Social Media Survey Apps (SMSAs)

 

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

c) offer something back to the user as an incentive to share their data (e.g. analysis or financial incentives)

Social Media Survey Apps (SMSAs)

 

A web or mobile-based tool built by a researcher in order to:

a) collect public and/or private data produced by social media users from an API;

b) collect supplemental information from such users (e.g. demographics) using more conventional survey methods;

c) offer something back to the user as an incentive to share their data (e.g. analysis or financial incentives)

For additional information, see:

Bail, Christopher A. 2015. “Taming Big Data: Using App Technology to Study Organizational Behavior on Social Media.” Sociological Methods and Research: 1-29

Workflow of a Social Media Survey App

Workflow of the Viralgorithm

Response Rate

 

  1. Approximately 40% of all organizations contacted installed the app (across two fields)

Minimal Evidence of Selection

Examples

Challenges of Building Apps for Social Science Research

 

  1. Significant coding skills required (html, css, cloud-computing, reactive programming)

  2. Competitive environment for attention (apps are no longer “new”)

  3. Concerns about data sharing/privacy

  4. Compelling incentives are hard to identify- and particularly challenging for studies of sensitive topics. But financial incentives may be an important option going forward.

Ok, but how do I build an App?

Shiny

Shiny is a (relatively) new tool that enables people to build, compile, and host interactive apps natively within RStudio

A Simple Shiny App

Three Components of A Shiny App

 

Global.R
server.R
ui.R

Global.R

 

-The “memory” of the app. Stores things you want to call from other parts of the app (usually data but could also be functions and other types of objects)

server.R

 

-The “brains” of the app- runs the analysis you want to show the user, but can also store data generated by the user, or expose different users to different types of information (good for experimentation)

ui.R

 

-The “face” of the app. Determines what user will see (e.g. what types of visualizations, check boxes or word boxes, fonts, etc.) Can load fancy images, logos, etc. to improve the overall appeal of the app.

Anatomy of a Shiny App

 

Shiny Templates

 

There are now many Shiny apps out there. Check out the templates (with code) here.

Linking to Cloud Servers

 

-Check out the googledrive package for loading and storing data.

-High-volume app hosting is available via RStudio.

Part 2: BOTS

Why Bots?

Anti-Racism Bots

Anti-Racism Bots

Using Bots to Study Social Media & Polarization

Using Bots to Study Social Media & Polarization

Using Bots to Study Social Media & Polarization

Using Bots to Study Social Media & Polarization

How do I Build a Bot?

A very simple bot

for (i in 1:24){
  #Search for 50 recent tweets about computational social science
  css_tweets<-search_tweets("Computational Social Science", n=50, include_rts = FALSE)
  #Randomly pick one of them, which appears in the `text` variable with the `css_tweets` dataframe
  lucky_tweet<-sample(css_tweets$text, 1)
  post_tweet(lucky_tweet)
  Sys.sleep(3600)
  #3600 seconds is 60 minutes
}

Making a bot interactive

 

Hosting a Bot

Hosting a Bot

 

Storing Data

 

Ethical Issues in App & Bot-Based Research

Ethical Issues in App & Bot-Based Research